Semiparametric posterior limits

We review the Bayesian theory of semiparametric inference following Bickel and Kleijn (2012) and Kleijn and Knapik (2013). After an overview of efficiency in parametric and semiparametric estimation problems, we consider the Bernstein-von Mises theorem (see, e.g., Le Cam and Yang (1990)) and generalize it to (LAN) regular and (LAE) irregular semiparametric estimation problems. We formulate a version of the semiparametric Bernstein-von Mises theorem that does not depend on least-favourable submodels, thus bypassing the most restrictive condition in the presentation of Bickel and Kleijn (2012). The results are applied to the (regular) estimation of the linear coefficient in partial linear regression (with a Gaussian nuisance prior) and of the kernel bandwidth in a model of normal location mixtures (with a Dirichlet nuisance prior), as well as the (irregular) estimation of the boundary of the support of a monotone family of densities (with a Gaussian nuisance prior).Comment: 47 pp., 1 figure, submitted for publication. arXiv admin note: substantial text overlap with arXiv:1007.017

Recovery, detection and confidence sets of communities in a sparse stochastic block model

Posterior distributions for community assignment in the planted bi-section model are shown to achieve frequentist exact recovery and detection under sharp lower bounds on sparsity. Assuming posterior recovery (or detection), one may interpret credible sets (or enlarged credible sets) as consistent confidence sets. If credible levels grow to one quickly enough, credible sets can be interpreted as frequentist confidence sets without conditions on the parameters. In the regime where within-class and between-class edge-probabilities are very close, credible sets may be enlarged to achieve frequentist asymptotic coverage. The diameters of credible sets are controlled and match rates of posterior convergence.Comment: 22 pp., 2 fi

The semiparametric Bernstein-von Mises theorem

In a smooth semiparametric estimation problem, the marginal posterior for the parameter of interest is expected to be asymptotically normal and satisfy frequentist criteria of optimality if the model is endowed with a suitable prior. It is shown that, under certain straightforward and interpretable conditions, the assertion of Le Cam's acclaimed, but strictly parametric, Bernstein-von Mises theorem [Univ. California Publ. Statist. 1 (1953) 277-329] holds in the semiparametric situation as well. As a consequence, Bayesian point-estimators achieve efficiency, for example, in the sense of H\'{a}jek's convolution theorem [Z. Wahrsch. Verw. Gebiete 14 (1970) 323-330]. The model is required to satisfy differentiability and metric entropy conditions, while the nuisance prior must assign nonzero mass to certain Kullback-Leibler neighborhoods [Ghosal, Ghosh and van der Vaart Ann. Statist. 28 (2000) 500-531]. In addition, the marginal posterior is required to converge at parametric rate, which appears to be the most stringent condition in examples. The results are applied to estimation of the linear coefficient in partial linear regression, with a Gaussian prior on a smoothness class for the nuisance.Comment: Published in at http://dx.doi.org/10.1214/11-AOS921 the Annals of Statistics (http://www.imstat.org/aos/) by the Institute of Mathematical Statistics (http://www.imstat.org

The Bayesian Analysis of Complex, High-Dimensional Models: Can It Be CODA?

We consider the Bayesian analysis of a few complex, high-dimensional models and show that intuitive priors, which are not tailored to the fine details of the model and the estimated parameters, produce estimators which perform poorly in situations in which good, simple frequentist estimators exist. The models we consider are: stratified sampling, the partial linear model, linear and quadratic functionals of white noise and estimation with stopping times. We present a strong version of Doob's consistency theorem which demonstrates that the existence of a uniformly $\sqrt{n}$-consistent estimator ensures that the Bayes posterior is $\sqrt{n}$-consistent for values of the parameter in subsets of prior probability 1. We also demonstrate that it is, at least, in principle, possible to construct Bayes priors giving both global and local minimax rates, using a suitable combination of loss functions. We argue that there is no contradiction in these apparently conflicting findings.Comment: Published in at http://dx.doi.org/10.1214/14-STS483 the Statistical Science (http://www.imstat.org/sts/) by the Institute of Mathematical Statistics (http://www.imstat.org

Analysis and improvement of the vector quantization in SELP (Stochastically Excited Linear Prediction)

The Stochastically Excited Linear Prediction (SELP) algorithm is described as a speech coding method employing a two-stage vector quantization. The first stage uses an adaptive codebook which efficiently encodes the periodicity of voiced speech, and the second stage uses a stochastic codebook to encode the remainder of the excitation signal. The adaptive codebook performs well when the pitch period of the speech signal is larger than the frame size. An extension is introduced, which increases its performance for the case that the frame size is longer than the pitch period. The performance of the stochastic stage, which improves with frame length, is shown to be best in those sections of the speech signal where a high level of short-term correlations is present. It can be concluded that the SELP algorithm performs best during voiced speech where the pitch period is longer than the frame length

Dependencies and Simultaneity in Membrane Systems

Membrane system computations proceed in a synchronous fashion: at each step all the applicable rules are actually applied. Hence each step depends on the previous one. This coarse view can be refined by looking at the dependencies among rule occurrences, by recording, for an object, which was the a rule that produced it and subsequently (in a later step), which was the a rule that consumed it. In this paper we propose a way to look also at the other main ingredient in membrane system computations, namely the simultaneity in the rule applications. This is achieved using zero-safe nets that allows to synchronize transitions, i.e., rule occurrences. Zero-safe nets can be unfolded into occurrence nets in a classical way, and to this unfolding an event structure can be associated. The capability of capturing simultaneity of zero-safe nets is transferred on the level of event structure by adding a way to express which events occur simultaneously

The Horseshoe Estimator: Posterior Concentration around Nearly Black Vectors

We consider the horseshoe estimator due to Carvalho, Polson and Scott (2010) for the multivariate normal mean model in the situation that the mean vector is sparse in the nearly black sense. We assume the frequentist framework where the data is generated according to a fixed mean vector. We show that if the number of nonzero parameters of the mean vector is known, the horseshoe estimator attains the minimax $\ell_2$ risk, possibly up to a multiplicative constant. We provide conditions under which the horseshoe estimator combined with an empirical Bayes estimate of the number of nonzero means still yields the minimax risk. We furthermore prove an upper bound on the rate of contraction of the posterior distribution around the horseshoe estimator, and a lower bound on the posterior variance. These bounds indicate that the posterior distribution of the horseshoe prior may be more informative than that of other one-component priors, including the Lasso.Comment: This version differs from the final published version in pagination and typographical detail; Available at http://projecteuclid.org/euclid.ejs/141813426

Codebook-based Bayesian speech enhancement for nonstationary environments

In this paper, we propose a Bayesian minimum mean squared error approach for the joint estimation of the short-term predictor parameters of speech and noise, from the noisy observation. We use trained codebooks of speech and noise linear predictive coefficients to model the a priori information required by the Bayesian scheme. In contrast to current Bayesian estimation approaches that consider the excitation variances as part of the a priori information, in the proposed method they are computed online for each short-time segment, based on the observation at hand. Consequently, the method performs well in nonstationary noise conditions. The resulting estimates of the speech and noise spectra can be used in a Wiener filter or any state-of-the-art speech enhancement system. We develop both memoryless (using information from the current frame alone) and memory-based (using information from the current and previous frames) estimators. Estimation of functions of the short-term predictor parameters is also addressed, in particular one that leads to the minimum mean squared error estimate of the clean speech signal. Experiments indicate that the scheme proposed in this paper performs significantly better than competing method